Symbolic Dynamic Programming within the Fluent Calculus

A symbolic dynamic programming approach for modelling first-order Markov decision processes within the fluent calculus is given. Based on an idea initially presented in [3], the major components of Markov decision processes such as the optimal value function and a policy are logically represented. The technique produces a set of first-order formulae with equality that minimally partitions the state space. Consequently, the symbolic dynamic programming algorithm presented here does not require to enumerate the state and action spaces, thereby solving a drawback of classical dynamic programming methods. In addition, we illustrate how conditional actions and specificity can be modelled by the approach.